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Abstract 



We study an AMOC time series model with an abrupt change in the mean and 
dependent errors that fulfill certain mixing conditions. We obtain confidence inter- 
vals for the unknown change-point via bootstrapping methods. 
Precisely we use a block bootstrap of the estimated centered error sequence. Then 
we reconstruct a sequence with a change in the mean using the same estimators 
as before. The difference between the change-point estimator of the resampled se- 
quence and the one for the original sequence can be used as an approximation of 
the difference between the real change-point and its estimator. This enables us to 
^ construct confidence intervals using the empirical distribution of the resampled time 

series. 

I A simulation study shows that the resampled confidence intervals are usually closer 

^ to their target levels and at the same time smaller than the asymptotic intervals. 
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1 Introduction 



Recently a number of papers has been published on possible application of bootstrap- 
ping or permutation methods in change-point analysis, confer Huskova ^JTj for a recent 
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1 Introduction 



survey. Most of these papers are concerned with obtaining critical values for the corre- 
sponding change-point tests. Another important issue in change-point analysis, however, 
is how to obtain confidence intervals for the change-point. In this paper we construct 
bootstrapping confidence intervals for the change-point in a model with dependent data. 

We consider the following At-Most-One-Change (AMOC) location model 

X{i) = fi + dl{iyrn}+e{i), 1 ^ i ^ n, (1.1) 

where m = m{n) = \n-d\, d = dn may depend on n. The errors {e(z), 1 ^ i ^ n} are 
stationary and strong-mixing with a rate specified below, 

Ee(i)=0, < o-^ = Ee(i)^ < oo, E |e(«)|'' < oo for some i/ > 4, 
^|cov(e(0),e(/i))| < cx), (1-2) 

and 

:= var e(0) -h2^cov(e(0),e(/i)) < oo. (1.3) 



The purpose of this paper is to develop and study a bootstrap suitable for getting 
approximation of the distribution of the following class of change-point estimators 

m(7) = argmax{|S'^(/c)|;/c = 1,. . . ,n - 1} 

= min{A;; 1 ^ < n, S^{k) ^ S-y{j), j = 1, . . . , n — 1}, 



where 

k 



n 

k{n — k) 



^ i=i 



and X„ = n Y^'l=iX{i). 

There is a quite extensive literature concerning asymptotic behavior of change-point 
estimators for independent observations. For a survey of various results, see e.g. Dumb- 
gen [13] , Csorgo and Horvath pj3] and Antoch et al. [1] . One of the first papers to derive 
the limit distribution for m(l/2) and independent errors under local changes has been 
written by Bhattacharya and Brockwell [7J. Diimbgen p3] considered a change in a gen- 
eral model for AMOC with independent observations and developed a suitable bootstrap. 
Antoch et al. [4 studied the asymptotic behavior of S^{k), < 7 < 1/2, in the model 



(1.1) with independent identically distributed errors and developed and studied a boot- 
strap valid for local changes. They also obtained various related results, such as rates 
of consistency for the estimators and their limiting distribution. Ferger and Stute [TC] 
and Ferger [T3l|T5] studied change-point estimators based on [/-statistics for i.i.d. errors. 



Bai [5] and Antoch et al. [3] analyzed the limit behavior of various estimators when the 
error sequence forms a linear process. However, they have not discussed bootstrapping. 



Most of the theoretical results concerning bootstrap methods in change-point analy- 
sis (testing and estimation) have been obtained for independent observations, see e.g.. 
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Huskova [17^ . Antoch and Huskova f2| obtained critical values for the change-point test 
related to functional of S^{k) iov Hq : m = n ("no change") vs. Hi : m < n,d ^ 
("there is a change in the mean") using permutation methods (or equivalently, boot- 
strap without replacement) for the independent case. Recently, Kirch |18| [T9l [20] has 
developed various bootstrap approximations for critical values for the above tests of 
"no change" versus "there is a change in the mean" suitable for the case of dependent 
observations that form a linear process. The results in |19j can also be modified in a 
straightforward way for dependent observations as discussed here. 

In this paper we develop and prove the validity of a circular overlapping block bootstrap 
for obtaining asymptotically correct confidence intervals in the case of dependent errors. 

In order to prove validity of the developed bootstrap scheme as well as to obtain the 
asymptotic under the null hypothesis for the change-point estimator we have to use 
some results like laws of (iterated) logarithm or large numbers for a triangular array. 



Therefore we need additionally to the assumptions (1.2) the following one for certain 
5, A > (in some cases > 2): 



{A) Let {e(i) : z G Z} be a strictly stationary sequence with Ee(0) = 0. Assume there 
are 5, A > with 

E|e(0)|2+'^+^ ^ Di 

and 

oo 

X;(fe + l)'/V(fe)^/('+'+^)<I)2, (1.5) 

where an{k) is the corresponding strong mixing coefficient, i.e. 
an{k) = sup \P{A nB)- P{A)P{B)\ , 

A,B 

where A and B vary over the cr-fields ^(e(0), e(— 1), . . .) respectively 
A{e{k),e{k + 1),...). 



Under this assumption we get moment inequalities (cf. Yokoyama [26], Theorem 1, and 
Serfling [21], Lemma B, Theorem 3.1), which in turn yield laws of large numbers. More- 
over the results remain true for triangular arrays that fulfill uniformly the assumptions 
above. For more details we refer to Kirch [18 , Appendix B.2. 

In fact we only need this assumption in order to obtain a Donsker type central limit 
theorem for the partial sums of the errors (to derive the asymptotic under the null 
hypothesis) as well as bounds on higher order moments of certain sums of the observed 
error sequence. This in turn yields laws of large numbers and laws of (iterated) logarithm. 
The proofs can easily be adapted to allow for errors that do not fulfill condition (A) but 
the necessary moment conditions. 

Example 1.1. Suppose that the errors form a linear process 
= "^Wseii - s), 
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2 Limit Distribution and Rate of Consistency for the Estimators 

where the innovations {e(i) : — cxd < i < 00} are i.i.d. random variables with 

Ee(z)=0, < o-^ = Ee(z)^ < 00, E |e(z)|'' < 00 for some > 4. 
We suppose that the weights {ws : s ^ 0} satisfy 

\ws\ < 00, / 0. 

Corollary 4 in Withers [25] gives mild conditions under which linear sequences are strong 
mixing and even provides the mixing coefficients. This can be used to check condition 
(„4). Causal ARMA sequences with appropriate innovations, for example, fulfill it for 
any (5, A > 0, if the {2 + 5 + A)-moment of the innovations exists. 

For the sake of simplicity we will only consider the case 7 = 1/2 in the following. The 
results for ^ 7 < 1/2 can be obtained in a similar way as outlined in Antoch et al. [4J. 
In the simulation study we will also consider other choices of 7, since the asymptotic 
method does not give such good approximations for 7 < 1/2. The reason is that the 
asymptotic distribution in this case depends on unknown parameters and thus in practice 
on estimators. 

In the present paper we focus on local alternatives (i.e., d = — > 0, as n — > 00). To 
obtain results for fixed alternatives is more complicated because the limit distribution of 
the estimator is determined by finite sums, which depend on the underlying distribution 
function. Some comments concerning the i.i.d. case can be found in the survey paper 
by Antoch and Huskova [1]. Furthermore Diimbgen |T3] considers both, local and 
fixed changes, for independent observations in a somewhat more general setup, i.e. the 
parameters that are subject to change need not be location parameters. 

In the following in := m(l/2), S{k) := 5'i/2(^)- 



2 Limit Distribution and Rate of Consistency for the Estimators 

In this section we summarize and generalize some previous results by Antoch et al. pi S] 
that we need in the sequel. 

The next theorem gives the rate of consistency for the change-point estimator m as well 
as its limit distribution for a local change. For the i.i.d. case these results have been 
obtained by Antoch et al. |4i. Theorem 1 and 2. The second result has been generalized 
for errors that form a linear process under the additional assumption ^3>o \ws\ < 00 
by Antoch et al. [3] (Theorem 2.2). 



Theorem 2.1. Assume that ( |l.l[ )-( 1.3 ) with < i? < 1 are satisfied and, as n ^ 00, 



dn^O, |d„rin-i/2(logn)i/2 = o(l). (2.1) 
Moreover let assumption (A) be fulfilled for some 5, A > 0. Then: 

a) The following rate of consistency holds, as n ^ 00, 
m — m = o{d~'^ log n) P — a.s. 
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2 Limit Distribution and Rate of Consistency for the Estimators 



b) The change-point estimator fulfills, as n ^ oo, 



T 



2 



aigmax{W{t)-\t\/2:t£R}, (2.2) 



where {W{t) : t G M} is a two-sided Wiener process and r as in (1.3). 



The proof is postponed to Section |5j 

Remark 2.1. We would like to point out that the result in a) also remains true for a 
fixed change, precisely it suffices to have \dn\~^ n~^/'^ (log n)^/"^ = o(l) we do not need 
dn — > 0. As a contrast we do need ci„ ^ to obtain the limit distribution in b), but if 
this is not fuUfilled the proof still shows fh — m = Op{d~'^). 



Remark 2.2. Also assertion (2.2) remains true, if one replaces the unknown quantity 



T by a consistent estimator r. It is also possible to replace dn by an estimator fulfilling 



dn — dn = Op I ylogn/n), since then it follows dn/dn 1 under (2.1). 



a) The following Bartlett type estimator is a consistent estimator for under the 



conditions of Theorem 2.1 (cf. Theorem 1.1 in Berkes et al. [6], Lemma A. 0.1 in 



Politis et al. [22], and Theorems 14.1 and 14.5 in Davidson [TT]): 

T^{An) = m + 2^1-— R{k), (2.3) 
fc=i ^ 

where 

^ / fh—k n—k 

^(^) = E - ^rn){Xt+k -Xfn)+ J] [Xt - xl){Xt+k - xl^ 



t=l t=m+l 

fh = min{argmax{|5(/i;)| : A; = 1, . . . ,n}} and = ZliLm+i Xi- An should be 
chosen such that A^/log(n) —>■ oo and n~^A^logA„ = o(log(n)). 

b) As an estimator for dn one can use 

^ n ^ fh 

dn = ^ ^(J)-^E^(J')- (2-4) 

j=m+l j=l 



Combining Theorem |2.l| a) with Theorem B.8 and Remark B.2 in Kirch [18j we obtain 
the rate 



dn-dn = o{\ P-a.s. 




under the assumptions of Theorem 2.1 



Remark 2.3. It can be shown that the above limit distribution is continuous and explic- 
itly known (confer Remark 2.3 in Antoch et Huskova [1]). Thus the above theorem can 
be used to construct asymptotic confidence intervals, precisely (m — t / dnqu{oi/2),rh — 
?/dnqL{a/2)), where P{U{l/2) ^ qda/2)) = a/2, P{U{\/2) ^ qu{a/2)) = a/2 for 



;7(l/2) = [/(??, 1/2) as in Remark |2^ below. Note that [/(??, 1/2) does not depend on 
the unknown parameter i?. 
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3 Bootstrap approximations 

Remark 2.4. For ^ 7 < ^ the limit distribution depends on the unknown parame- 
ter ■!?. Precisely it can be shown that the limit is 

U{^,-f) = argmax{VF(t) - \t\g^,^{t)}, 

t<0, 

where as ^(i = < 

\(l-i9)7 + i9(l-7), t^O. 

Remark 2.3 in Antoch et Huskova [1 gives a closed formula for the limit distribution. 
We would like to point out a small (but for simulations very important) misprint there: 
The integral is equal to 02/(01 + 02) + ■ ■ ■ and not to ci/(ci + 02) + ■ ■ ■■ 



3 Bootstrap approximations 



Antoch et al. propose a bootstrap with replacement of the estimated error sequence to 
obtain confidence intervals for the change-point. Since in our case the error sequence is no 
longer independent we have to use a slightly different approach here. We still bootstrap 
the estimated error sequence with replacement, but we will now use a circular moving 
block bootstrap as suggested by Politis and Romano |23]. It has the advantage over 
the regular moving blocks bootstrap by Kiinsch [21] that the sample mean is unbiased. 
Another possibility is to use non-overlapping blocks as suggested by Carlstein [9], but 
this bootstrap does behave slightly worse in simulations. 

Kirch |18| [T9] used block bootstrapping procedures (more precisely a block permutation 
method as well as a circular and non-circular block bootstrap) to get approximations for 
the critical values of the change-point test corresponding to the above problem. 

Block bootstrapping methods split the observation sequence of length n into sequences 
of length K. Then we put L of them together to a bootstrap sequence (i.e. n = KL). 
We keep the order within the blocks. K and L depend on n and converge to infinity 
with n. 

The idea is that, for properly chosen block-length the block contains enough informa- 
tion about the dependency structure so that the estimate is close to the null hypothesis. 

We assume in the following that 

K,L^ 00 and K = K{L), n = n{L) = KL, K/ L = o{l). (3.1) 



Let pi be an estimator for /i, p,2 for fd + dn, and dn for dn, e.g. 

^2 = V X{i), dn = 

m ^ — ' 77, — 777, ^ — ' 



1^1 



m 



1=1 



(3.2) 



i=m+l 



where rh = m(l/2) as in (1.4). Remark 2.2 yields that dn fulfills assumption (3.5). 
Define the estimated residuals and the centered residuals by 

e{i) = X{i) - /ill{i<jm} - /^2l{m<j^n}; 
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3 Bootstrap approximations 



1 " 

= e(i) - -^e{j), 



respectively. Throughout the paper the following representation will turn out to be very 
useful 



e(i) =e(i) -en + dn[ l{m<i^n} 



n — m 



n 



n — m 



n 



(3.3) 



Let (C/(l), . . . , U{L)) be i.i.d. with P{U{1) = i) = i for i = 0, . . . , n - 1 independent 
of the observations X(l), . . . , Take the i.i.d. bootstrap sample e*{Kl + k) = 

e{U{l) + fc), where e{j) = e(j — n) for j > n (hence the name circular bootstrap). 

Consider the bootstrap observations 

X*{Kl + k) = e*{Kl + k) + /ill{A'/+fc^m} + M2l{m<m+fc^n}- 

We now deal with the following bootstrap estimator of the change-point m 

fa* = argmax{|S'*(A;)|; A; = l,...,n — 1}, (3.4) 

where 

1/2 fe 



S*{k) 



n 



k{n — k) 



Now we are ready to present results on the asymptotic behavior of the bootstrap esti- 



mator m* defined in (3.4) of the change-point together with a short discussion, how to 



apply the result to obtain confidence intervals. 

With P*, E*, var*, ... we will denote probability, expectation, variance,. . ., given 
X(l),...,X(n). 



Theorem 3.1. Assume that (|1.1|)-(|1.3|) with < i? < 1 and (3.1) hold. Moreover let 

(3.5) 



dfi dfi — O 



log n 



P-a.s. 



n 



be fulfilled in addition to (2.1 ). Moreover let assumption (A) be fulfilled for some 

< (5(2) + a(2) < (5(1) - 2)/2 <{u- 4)/2, A(i) = i/ - 2 - 6^^\ (3.6) 
// dlK 0, then 



sup 



P*{d^{m* — fh) ^ x) — P{d^(jh — m) ^ x) 



P-a.s. 



Since the limit distribution (for both the bootstrap as well as null asymptotic) is contin- 



uous (as has been pointed out by Remark 2.3) the described sampling scheme provides 



bootstrap approximations to the (1 — a)-quantile for arbitrary a E (0,1). Thus the 
bootstrap based approximation for the change-point m can be constructed along the 
usual lines. Precisely the (1 — a)-bootstrap confidence interval is given by 

[2m-ql{a/2),2m-qUa/2)], 
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4 Simulation Study 

where 

ql{a/2) = sup{u; P* {in* < n) ^ a/2} 

and 

qu{a/2) = inf{u; P* {m* > n) ^ a/2]. 

Usually one uses the empirical bootstrap distribution of fh* for say 10 000 random boot- 
strap samples. Further discussions on bootstrap approximations of confidence intervals 
(for the similar case of i.i.d. errors) can be found in Antoch and Huskova [1]. 

Remark 3.1. There are also several other possibilities of bootstrapping. For example 
we can use a non-circular approach and/or non-overlapping blocks. Simulations for the 
bootstrap where . . • , U{L)) are i.i.d. uniformly distributed on {0, . . . , L — 1} and 

e*{Kl + k) = e{KU{l) + k) indicate that this bootstrap does not perform quite as good 
as the bootstrap proposed above. 



4 Simulation Study 

In the previous chapter we have established the asymptotic validity of the bootstrap 
confidence intervals. The question remains how well these confidence intervals behave 
for small samples and also how well they behave in comparison with the asymptotic 
intervals. 

In this section we not only consider 7 = ^ but also 7 = 0. The important difference 
is that the asymptotic confidence intervals depend on the unknown parameter for 
^ 7 < ^. Not surprisingly it turns out that the asymptotic intervals behave better for 
7=2) whereas in all other cases it is better to use the bootstrap intervals. 
Moreover we consider changes in the mean of (i = 0.5,1,2,4. The latter ones can hardly 
be regarded as local changes, however we are still interested in the behavior of the 
bootstrap intervals, since we conjecture it will also be valid in those cases. 

For the simulations we use an autoregressive sequence of order one as an error sequence 
with standard normally distributed innovations and different values of p = 0.1,0.3. We 



consider changes at m = 20,40. We use the estimator dn as in ( |2.4[ ) respectively (3.2) 
and - for the asymptotic method - the Bartlett estimator given in ( |2.3[ ) with A„ = 0.1 n, 
because in the simulation study conducted by Antoch et al. [3] this choice gave best 
results in the AR(l)-case. 

The goodness of confidence intervals can essentially be determined by two criteria: 

C.l The probability that the actual change-point is outside the (l-a)-confidence interval 
should be close to (smaller than) a. 

C.2 The confidence intervals should be short. 



We visualize the first quantity by using CoLe-Plots (Confidence-Level-Plots) and the 
second one by using CoIL-Plots (Confidence- Interval-Length-Plots). 

In fact we have done more simulations (such as QQ-plots or tables of the quantiles of m) 
for a large amount of different combinations of parameters as well as different possible 
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4 Simulation Study 



bootstrap procedures. The problem, however, is that for the bootstrap they only give 
result for one specific underlying sequence and are thus rather not as informative. For 
this reason and also due to similarity of results as well as due to limitations of space we 
restrict ourselves to the following plots. 

CoLe-Plots 

We explain how the plots are created using the example of asymptotic confidence inter- 
vals. The general version of Theorem |2 . 1 1 yields that the asymptotic confidence intervals 
are calculated using the distribution of Z = in — t /d^y(i?,7), where ^(^^,7) IS as m 
Remark 



2.4 



and ■!? = fh/n. Note that 



m0C/(l-a) P(Z^m)^a/2 or P(Z ^ m) ^ a/2 

2 min(P(Z ^ m),P(Z ^ m)) ^ Q. 

The CoLe-Plots now draw the empirical distributions function (based on 1 000 observa- 
tion sequences) of 2min(P(Z' ^ m), P{Z ^ m)). 

Thus for given a on the x-axis the plot shows the empirical probability that m is outside 
the (1 — a)-confidence interval on the y-axis, hence it visualizes C.l. Optimally, the plot 
should be below or (even better) on the diagonal. 

For the bootstrap confidence intervals the procedures works exactly the same but now 
the intervals are calculated using the (empirical, based on 10 000 resamples) distribution 
of .Z' = 2fh — fh* . 

CoIL-Plots 

We calculate for 1 000 observation sequences the length of the confidence intervals for 
levels a = 0.01, 0.02, 0.1. The empirical bootstrap distribution is based on 10000 
random samples as before. Then we plot the mean using a thick line (as well as the 
upper and lower quartiles with thin lines), linearly interpolated. So these plots visualize 
the length of the intervals and thus C.2. 

Note that the scale on the y-axis is not the same for different pictures. This way we can 
better compare the asymptotic with the bootstrap method. 



The plots are given in figures 4.1(i)||47l(ii)| for 7 = and 4.2 for 7 = 0.5. Concerning 



the CoIL-Plots we only plot the means for better readability. In Figure 4.3 we give 



the CoIL-Plot corresponding to Figure 4.1(i) (2) including the quartiles to give a better 



idea of the distribution of the length of the confidence interval. Concerning 7 = we 
see that for small dn the actual cover probability of the interval is too small for both 
methods, yet the asymptotic interval is somewhat better than the bootstrap intervals. 
At the same time the length of the asymptotic interval is very large, much larger than 
the length of the bootstrap interval. Frequently it is even longer than the observation 
sequence. We did not correct upper and lower bounds of the intervals by respectively 
80, but bootstrap intervals can also be outside that possible range. 

In fact it is somewhat surprising that even though the intervals are quite long the levels 
are not as good. The reason is that the change-point estimator for such a small change 
(and relatively few observations points) is frequently not very good. A typical example 
is an observation sequence with a change at m = 20, where the estimator suggests a 
change at 70. This results in intervals that do not contain the actual change-point. 
Also this leads to a wrong estimation of the parameters of the underlying asymptotic 
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(1) CoLe-Plot: d = 0.5 
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(2) CoIL-Plot: d = 0.5 
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(3) CoLe-Plot: d = 1 
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(4) CoIL-Plot: d = 1 
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(5) CoLe-Plot: d = 2 




boot, 
asym. 



■ 


K=2 


■ 


K=4 


■ 


K=10 



O50 052 O04 O06 O08 OO 02 04 
(6) CoIL-Plot: d = 2 
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(7) CoLe-Plot: d = 4 
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(8) CoIL-Plot: d = 4 



Figure 4.1(i): CoLe- and CoIL-Plots for n = 80, m = 40, p = 0.1, and 7 = for 
the asymptotic method as well as bootstrap method with different block- 
lengths 



10 



4 Simulation Study 




1)1)5 oM 



0.15 



0.20 



(9) CoLe-Plot: d = 0.5, m = 40 
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(10) CoIL-Plot: d = 0.5, m = 40 
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(12) CoIL-Plot: d = 2, m = 40 
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(13) CoLe-Plot: d = 0.5, m = 20 



boot, 
asym. 
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(14) CoIL-Plot: d = 0.5, m = 20 
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(15) CoLe-Plot: d = 2, m = 20 



boot, 
asym. 
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(16) CoIL-Plot: d = 2, m = 20 



Figure 4.1(ii): CoLe- and CoIL-Plots for n = 80, p = 0.3, and 7 = for the asymptotic 
method as weh as bootstrap method with different block-lengths 
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(18) CoIL-Plot: d = 0.5 




o5o o55 oTo (TTs 0.20 
(19) CoLe-Plot; d = 1 
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(20) CoIL-Plot: d = 1 
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(21) CoLe-Plot; d = 2 
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(22) CoIL-Plot: d = 2 
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(23) CoLe-Plot: d = 4 



0.20 



boot. 

asym. 




■ 


K=2 


■ 


K=4 


■ 


K=10 



Oo 0l)2 O04 O06 O08 0?I0 0?I2 04 
(24) CoIL-Plot: d = 4 



Figure 4.2: CoLe- and CoIL-Plots for n = 80, m = 20, p = 0.3, and 7 = 0.5 for the 
asymptotic method as well as bootstrap method with different block-lengths 
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4 Simulation Study 




Figure 4.3: CoIL-Plot for n = 80, m = 40, p = 0.1, 7 = 0, and d = 0.5, which addition- 



ally to Figure 4.1(i) (2) includes quartiles of the interval length 



distribution, which is then highly skewed in the wrong direction. Thus the lower quantile 
of the interval is something around 60, whereas the upper quantile is far bigger than 80. 

For more obvious changes the level of the intervals as well as the length becomes better. 
This is somewhat surprising in case of the asymptotic intervals because for fixed changes 
the asymptotic is not valid. The reason is that we have an interval around the change- 
point estimator, which is quite good for more obvious changes. 

If the changes are closer to the border of the interval, the levels for both methods 
deteriorate somewhat. The same holds true for stronger correlation of the underlying 
error sequence. 

Overall the bootstrap intervals behave better than the asymptotic intervals. 

However, in the case of 7 = 0.5 the asymptotic distribution does not depend on unknown 
parameters anymore. In this case the asymptotic confidence intervals for local changes 
are in fact better than the bootstrap intervals. The levels of both methods are for small 
d somewhat worse than for 7 = 0, but the lengths are much better, especially for the 
asymptotic intervals. However, for more obvious changes the bootstrap intervals are 
again better than the asymptotic ones. This is due to the fact that the asymptotic does 
not hold in this case. 

It is worth noting that the performance of the bootstrap method does not seem to depend 
significantly on the choice of the block- length. This is in contrast to the situation where 
we bootstrap critical values for change-point tests (cf. Kirch |1^) where a larger block- 
length was needed when the data was more dependent. 
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5 Proofs 



In real-life situations we recommend to rather use the bootstrap intervals, since they 
work no matter what 7 and for both, local as well as fixed changes. 



5 Proofs 



Throughout the proofs we use the notation a„ <C bn for a„ = 0{bn). 
We start with the proof of Theorem 2.1 in Section [2j 



Proof of Theorem 2.1\ We only sketch the proof, because it is very similar to the 
proof of Theorem 1 respectively 2 in Antoch et al. [1] . First note that 

rh = argmax{y(/c) : k = 1, . . . ,n — 1}, where V{k) = S{k)'^ — S{rn)^ . 

Simple calculations yield for k < m 

n{m — k){n — m — k) 



V{k) 



^ k \ 

^(e(i) - en) 

\i=l / 



k{n — k)m{n — m) 

m 

^ (e(i) - e„) ( ^(e(i) - e„) + ^(e(i) - e^) 



n 



m{n — m) 



i=k+l 



i=l 



2dn ^ (e(i) - e„) 

i=k+l 

+ 2dn -Y.^e{i) - 



(5.1) 



n — k 

1=1 

,2 (n — m){k — m) 



+ '^" n-k 

First we show assertion a), i.e. the a.s. rate of consistency for the change-point estimator. 
Theorem B.8 b) and Remark B.2 in Kirch [18; give 







max 


Ak5 


k^m 


= o{dJn 



_2 n\n — m — k\ 
k^m-an " m(n — m)^ 



max d, 



1 

1=1 



Similarly we get for a„ = ed^^logn, where e > is an arbitrary fixed constant, 

Ak2 



max 

k^m—an 



A 



k5 



k^m-an m{n — my 



m — k 



i=k+l 



Y^ieii) - en) + 



i=l 



i=l 

P — a.s., 
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max 



Ak3 



max 



Ak5 

a 



-1 



A, 



fc5 



max 2d 

fcsCm— a„ 



-1 



n — k 



n — m 



(e(0 - en) 



m — k 



=k+l 



log n 



o(l) P-a.s., 

k 



max 2d 



-1 



1 



n — m 



I] - e. 



i=l 



log n 



o(l), P-a.s. 



Note that is increasing in k for k ^ m, so that 



max^fcs = and max Ak5 ^ — elogn. 

kS^m k^m—a„ 



Thus 



P (m ^ m — ed^ log n inf. often) 



^ P I max ^ max inf. often 

\^ki^m—a„ k>m—a„ 

^ P ( max y(/c) ^ inf. often ) 



^ P max {Afc5 (1 + ^ inf. often 
= 0. 



with Y„ = 0(1) P — a.s. 



A similar argument gives 

P [jh ^ m — ed^ log n inf. often) = 0. 
Hence assertion a) is proven. 

For assertion b) we first need somewhat stronger bounds for the above sums, but only 
in a P-stochastic sense. Theorem B.3 in Kirch [18 gives a Hajek -Renyi type inequality 
if certain moment conditions of the sums are fulfilled. This yields here (C > arbitrary 
fixed constant) 



max 



max 



1 " 

^&a)-e-„) 

1 " 

y](e(i) - e„; 

, /r) ^— ' 



op( v^logn), 
Op(l), 



max 

ki^m—CdZ, 



m — k 



j=k+i 



Op(l), 
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where the last hne follows because for all 7n ^ 1 (in particular for 7^ 00), D > and 
some 5 > 



P In'^ max 



i=i ) 

\k=\ fc=7„ + l / 



1 



Analogously to above this yields for 6„ = Cd~'^, where C > is an arbitrary fixed 
constant, 



max 

k-S^m—b„ 



max 

k^m—bn 



A 



kl 



Ak3 



A 



k5 



op{l), 
C-i/20p(l 



max 

ki^m—b„ 



A 



k2 



Ak5 



max 

k^m—b„ 



--opil), 



op{l). 



Similarly for —hn^k — ra<{) 
max l^fcil = op(l), 

— 6„ ^fc— m<0 

op(l), 



max \Ak2\ 

— bn ^k~m<0 



op{l), 



max \Ak4\ 

-bn^k—m<0 



A 



fc5 



-{m - k)d^ + op(l), 



where the last rate is uniformly in — ^ k — Tti <i 0. The proof can be finished 
analogously to the proof of Theorem 2 in Antoch et al. 14, , where we now use Theorem 1 
of Section 1.5 in Doukhan 1121. ■ 



We will first formulate some auxiliary lemmas, which will enable us to prove the results 
in Section [3l 

Lemma 5.1. Let ^n(l)i • • • ,Cn{n) be a triangular array of row-wise i.i.d. random vari- 
ables with E^„(l) = and E^^(l) = o"^ + o(l) as n ^ 00, then 



E ^"(•?') : ^ t ^ 1 i ""A^^ {W{t) -.O^t^l}. 



Proof. It is analogous to that of Theorem 16.1 in Billingsley [8], since the central limit 
theorem holds for triangular arrays and the proof of tightness also works analogously. ■ 



Lemma 5.2. Assume that (1.1) -(1.3) withO < < 1 and let (2.1) be fullfilled. Moreover 



let assumption {A) be fulfilled for some < 5 < (i^ — 4)/2, A = z^ — 4 — 2(5. If additionally 



(3.5), then 



1 



K 



var 



log n 



L 



t2 + o(1) = 0(1) P-a.s., 



where r. 



^ n— 1 



1=0 



1 ^ 

— E 



e{l + A;) 



and e{j) = e(j — n) for j > n. 
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Remark 5.1. More careful considerations concerning ^3 below even yield an almost 
sure rate of o(^). 

Remark 5.2. 'Estimator' TVt is closely related to the Bartlett window estimator with 
parameter K — 1 if for this estimator one also uses a circularly extended series, precisely 

1=1 t=i ^ ' 1=1 



Proof. For m < m (the other case can be dealt with in a similar way) (3.3) yields the 
following decomposition 



var 



2A, 



where 



1=0 / 



n-l / K 



1=0 \k=i 

n-l K 



Ket 



dn I l{n^Z+fc>m} 



n — m 



n 



dn I l{n^i+fc>m} 



n — m 



n 



1=0 k=l 

• ^ [dni l{n^«+A:>m} 

Theorem B.8 b) in Kirch [IE] yields 



n — m 



n 



A 



log n 
L 



dn I ^{n^l+k>m} 



P-a.s. 



n — m 



n 



Concerning A2 we have 

n-l 

L 



A2«jy2(d^ 



1=0 



n — m n — m 

- dn 



n 



n 



n-l / K 



iTif f {dn'^{n^l+k>m} " dnl{n^l+k> 



1=0 \k=l 

A21 + A22, 



m} 



now Theorem 2.1 a) and ( |3.5[ ) yield 

A2,^K(dl ^'^~^'^\ {dn-dn? 



n 



L 



P - a.s. 
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Similar arguments give 

n-l / K 



A 



22 



^ n-l / K \ 1 "^"^ / ^ \ ' 

;=0 \A;=1 / /=0 \k=l J 



Li 



o 



log n 
L 



P-a.s. 



Note that 



E 



\ 2 

1 \ 1 

-/^ E = var(e(0)) + 2-^5] cov(e(i), e(j)) 

i=i y «<j 



var(e(0)) + 2 cov(e(0), e{j)) ( 1 - ^ 



t' + o(1), 



where the last line follows because of the absolute summability of the covariance function. 
Thus, Ai = t'^ + o(l) = 0(1) P - a.s. because of (3.6.7) in Kirch US]. We do not 
need assumption (3.6.6) there because (3.6.9) can be strengthened using the Minkoswki 
inequality as follows: 



n-l 



-En 



i=0 



<E 



k=l 1=0 



K L-1 

k=l 1=0 
2+5 



^2+5 



/ ^ L-1 2+5\ 
k=l \ 1=0 J 



2+s\ l/(2+5)' 



2+5 



_^-{2+5)/2^ 



The Cauchy-Schwartz inequality yields 



^3 ^ ^/AlA2 = O 



log n 



L 



P-a.s. 



Putting everything together we arrive at the assertion. 



Lemma 5.3. Assume that (1.1)-(1.3) withO < ?? < 1 and let (2.1) and (3.5) be fuUfiUed. 



Moreover let assumption (A) be fulfilled for some 5,A>0. Then, 

K 



1 



max 



i^KL ^/Kl + 1 
l<k<K 



j=k+i 



max 



i^KL yjKl + 1 

l<k<K 



K 



E {<U{l)+j)--en) 

j=k+l 
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max 

l<k<K 



max 
l^k<K 



Ly^e*(i) = max ^ V [e(?7(0) + i) - eJ 
1=1 1=1 



1 



^/K -k 



i=k+l 



max 



max , 

l<k<K 



max 



K 



j=k+i 



l<k<K 



K 



E ieiUil)+j)--er, 
j=k+i 



with \Rn\u)\ ^ iinj^ = 152,3, where Rn does not depend on {U{-)} and 
Rn = o (^^/logn^ P-a.s., 

and Rn\u) ^ Rn, where Rn does not depend on {U{-)} and Rn = o(l) P — a.s. 
If additionally Kd^ = o(l), then Rn = o(l) P — a.s. 



Proof. We only prove the first result, the others can be proven similarly. Let m < in, 



the other case is analogous. By (3.3) we get 

K 



1 

max , = 
liiKL ^/Kl + l 
l<k<K 



^ max 



j=k+i 

K 



max 



i^KL ^JKI + 1 
l<fc<A' 



K 



E {e{U{l)+j)--en) 
j=k+i 



l^KL y/Kl + 1 
l<k<K 



j=k+l 



i^U{l)+k>m} 



n — m 



n 



dn ( ^{n^U{l)+k>fh} 



n — m 



n 



<C max 



1 



l,k ^/Kl + 1 



K 



j=k+l 



El n — m -r^ n — m 
I dn dn 



n 



n 



1 



max 



l,k y/Kl + 1 

1 

+ max , = 

l,k yjKl + 1 

Bi + B2 + B'i, 



K 

dn^{m.<U{l)+k<ifi] 

j=k+l 
K 

E \dn — dn\l{n^U{l)+k>ffi} 
j=k+l 



where by Theorem 2.1 a) and (3.5) we get (note that min(A', \m — fh\) ^ 



m — m\ 



B,^VK{\dn\^^^^-^ + \dn-dn\] = O ( ^ ^ ] = o(l) P - U.S., 
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B2 <C mm{K, \ m — m\) \dn\ ^ min (\'^\dn\, \/|m — "^1 

P — U.S., 



a ( min ( \/log n, \fKdr 



Bs « VK\dn-dn\ =0 



This gives the assertion. 



log 71 

L 



0(1) P-a.s. 



Lemma 5.4. Assume that (1.1)-(1.3) withO < < 1 and let (2.1) and (3.5) be fullfilled. 



Moreover let assumption (A) be fulfilled for some < 6^"^^ + A^^^ < (5^^^ — 2)/2 < 
(jy - 4)/2, A(i) = 1/ - 2 - (^(1) . T/ien, 



a) 



1 1 

^^^= max 



Op.(l) p-a.s. 



6) max 



i=l 

a 



Op.(l) P-a.s. 



// moreover d^K 0, t/ien 

1 



d-'c'/' 



max 



l^j^fh-Cd~^ fn — J 



Op.(l) p-a.s. 



Proof. The proof of Lemma 5.2 shows that 



E* 



-=^(e(C/(0) + A;)-e„) = 0( 
fc=i / 



1) P-a.s. 



(5.2) 



Analogously we get from the proof of Theorem 3.6.2 and Remark 3.6.4 in Kirch |18] for 
some (5 > 



E* 



E* 



max 
k=0,...,K~l 



(log n) 1+^/2 i^fc^A 



^ ^ (e(C/(0)+j)-e-„) 

1 ^(e(C/(0)+j)-e-„) 



0(1) P-a.s. 



max 



(5.3) 



0(1) P-a.s. 



Thus we get by Lemma 5. 3| using the Markov resp. Hajek -Renyi inequalities 

3 



P max 



< P* max 



1=1 



^ C0Og 



n 



1 



i^KL ^Kl + 1 



+ max 



5^^e*(i^. + A:) 



j=0 A;=l 



+ max —p^ 



li^KL yjKl + 1 



K 



{e{U{l)+j)-en) 

j=k+i 



Y^{e{U{0) + i)-en) 

i=l 

^ c-y/log n 
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log n 
+ E 



1=1 \V-f^ / 

1 ^(e(f/(0) + i) - e-„) 

i=l J 

k 

1 ^(e(f/(0)+j)-e-. 



^ (log n)i+V2 1^/1+5/2 



max 



V 



lvar*(^^e*(A;)^+o(l) = 0(l) P 



a.s., 



where the last line follows from Lemma 5.2 This is assertion a), assertion b) is analogous. 
Concerning assertion c) we have: 

1 



P* d-^C^I"^ max 



i=j+i 



i<:fh-cd-'^ m- J 
« P* (d;iCi/2|g* I ^ ^/^ ^ j 1^1/2 

+ P* i^n^C^'^ 
+ P* I d-^C^/'^ 



^ c 



1 



max 



l=\j/K^ k=i 



max 



1 



max 



m 

\ j=K[m/Ki+l 
D1 + D2 + D3 + D4 + D5. 





K 


1 


E ^- 


- {Kl + k) 


K\j/K] 













^ c' 



The Chebyshev inequality, Lemma 5.2, and the fact that d^^n = o(l) P — a.s. yield 



Z)i = P*(^d^iC^/2|e;| ^c') ^Cd-^n-i-^var* ^^e*(i)^ = o(l) P - a.s. 
Moreover the Hajek -Renyi inequality gives (note that Yl^=an ^/^'^ ~ 0(l/a„)) 

[m/A'J-l 



/ K \ lm/K\-l 

^'==1 ^ L l=Cd-^/K+l 

< var* e*{i)^ = 0(1) P - a.s. 



+ 



1)2 ^ Ci^ 
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Since Kd!^ and by Lemma 

(m-Cd-^)/K 



5.3 



we get 



D:^ « 



1=1 



C71/2 



d„ m-Kl-K 



K 



max > e*(Kl + 



■ E 



2+5 



{m-Kl- K) 



2+5 



E* 



1 



K 



max ^ V (e(C/(0) + j) - e" 
j=fc+i 



2+5 



2+5 1 



^2+5 



m/K-2 

E 



1 

^•2+5 



j=Cd-^/K 

<^ (C'^dlKy^^ = o{l) P-a.s. 
The arguments for 1)4 and are analogous and therefore omitted. ■ 
Now we are able to prove the result for the local case. 

Proof of Theorem 3.1[ The proof is very close to the proof of Theorem 3 of Antoch 
et al. [4 , therefore we will only sketch it. 

Taking x > 0, without loss of generality, we have for large enough C > 

P* (S^nim* - a) ^ =P* (m - Cd~^ ^ m* ^ m + xd'^) 

+ P* (^diim* -m)^x, \m* - m\ > Cd~'^ 
and the second term is smaller than 

P*{m* <m- Cd;^^) + P* (m* >fh + Cd'"^ 



Similar to the proof of Theorem |2.1| we will now show that this becomes arbitrarily small 
for almost all samples X(l), . • • First note that by Theorem 2.1 a) there exists 

e > with 



m 

< e < — < 1 - e 
n 



P-a.s. 



We have an analogous decomposition V* = A*^ + A* 2 + A*.^ + A*^ + A*^ as in (5.1) 
where we replace m hy fh and e(-) by e*(-). 



Then, by Lemma 5.4 



max_ 



A* 



A* 



7-2 -1 1 
<^ d^ n max - 



i=l 



Op* ( d„ n log n 



= op*(l) P — a.s., 
since d^^n^^^'^ilogn)^^'^ P — a.s. Similarly we get 

1 



max 



ji^m—Cd„ 



A* 



A* 



<^ n ^^^d„ ^ max 



i=j+l 



1 




max —— 






i=l 



Op* (d-^n-^/^) = op* (1) P-a. 



s. 
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jS^fh—Cdn' 



A* 

A* 



^71 



1 



C^^''^Op*{l) P-a.s. 



max_ 



A 



A* 



<^ d„ max — = 



i=j+l 



1=1 



Op* (1) P — a.s. 



This gives a s in the proof of Theorem 1 in Antoch et al. [4 (and similar to the proof 
of Theorem 2.1) that P*{m* < fh — Cd~'^) becomes arbitrarily small P — a.s. Similar 
arguments hold true for P*(jh* > m + Cd~'^). 

So it suffices to consider (C large enough) 

P*{m-Cd~^ i^m* i^m + xd-^) = P* I rnax V* ^ rnax V*] . 



(j-m)d2(z[-c,x] 



{j-fh)dl^[-C,x] 



For —Cdj^'^ ^ j — in < we get similarly to above 



_max I 

-Cdn^^j-fh<0 



A*^\=Op* [d-^n~^\ =op*{l) P-a.s. 



max I At 



h\ = Op* {d-Jn~"\\ognr^'^) = op*{l) 



P — a.s. 



-Cdn^j-m<0 

max \A*\ = Op* (d-^n-^^^) = op*{l) P 

-Cdn^^-m<0 ^ ' 



a.s. 



Analogous arguments give the assertion for j > m. We can now essentially finish the 
proof as in Antoch et al. Note that P — a.s. we have 



Y* = < 



0, j = m, 



^-2d„^^^^^^e*(i) +d^(m- j) +op*(l), m<j i^m-xd-^, 
A similar argument as in Lemma |5.4| yields 

K 



1 



max 



E 

i=k+l 



e U 



Op*{l) P-a.s. 



Thus we get for m — Cd^ ^ ^ j < fh 

\rhlK\ 



Vm/K\ K 

V* = 2dnVK ^ \2e*{Kl + k)-dl{fh-j) + op*il), 



l=lj/K] fe=l 

and a similar equation in the other case. Consider the process 

'O, s = 0, 



^ lfh/K\ 



^ ^Y.k=i^*^Kl + k) + s/2, s<0, 



[(m+d„V2s)/ii'J 1 
l=\{m+l)/K^ 



Efc=ie*(K/ + fc)-s/2, s>0. 
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Asymptotically the maximum of is the same as the maximum over {T^(s)}sg]K. 



By Lemmas 5.1 and 5.2 the process {l^(s)}<jgiR converges P—a.s. to the process {VF(s)- 



|s|/2 : s G M}. Hence, as L — > oo, 



P* max V* ^ max V* 

\{j-fn)dl&[-C,x] {j-fn)dU[-C,x] ^ 



P* sup {W{s) - \s\/2) ^ sup {W{s) - \s\/2) 
\-c^s<:x se]x,c] 



= P* \^-C ^ argmax(Ty(s) - \s\/2) ^ x) P-a.s. 
Letting C ^ oo yields the desired result. ■ 
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